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Abstract 

In this paper we study ergodic backward stochastic differential equations (EBSDEs) drop- 
| ping the strong dissipativity assumption needed in [12] . In other words we do not need to 

. require the uniform exponential decay of the difference of two solutions of the underlying 

forward equation, which, on the contrary, is assumed to be non degenerate. 

We show existence of solutions by use of coupling estimates for a non-degenerate forward 
stochastic differential equations with bounded measurable non-linearity. Moreover we prove 
uniqueness of "Markovian" solutions exploiting the recurrence of the same class of forward 
equations. 

Applications are then given to the optimal ergodic control of stochastic partial differential 
equations and to the associated ergodic Hamilton- Jacobi-Bellman equations. 
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1 Introduction 

Since the beginning of the 90's several papers have described the link between backward stochas- 
tic differential equations (BSDEs), Hamilton- Jacobi-Bellman equations and stochastic optimal 
control (see, for instance, [8j and [E]). The successive literature on BSDEs covered several 
different situations, among them infinite horizon control problems, both in finite and infinite 
dimensions (see [3], [21] and |14J). 

In |12j the BSDE approach was extended to the case of ergodic control problems, that is, 
of control problems in which the cost functional only evaluates the long time behavior of the 
stochastic system. In that paper the authors introduced the following class of BSDEs with 
infinite horizon, called Ergodic BSDEs (EBSDEs): 

Yf =Y$+ f ty(X2,Z*) - \}da - f Z*dW a , P - a.s., V0 < t < T < oo, (1.1) 
Jt Jt 
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where (Wt)t>o is a cylindrical Wiener process in a Hilbert space S, X x is the solution of the 
following forward SDE with values in a Hilbert (in |12j also Banach) space H, 



dXf = (AX? + F(X?))dt + GdW t , X = x, 



(1.2) 



and ip : H x H* — > R is a given function. 

We underline that the unknowns in the above equation is the triple (Y, Z, A), where Y, Z are 
adapted processes taking values in R and S*, respectively, and A is a real number. 

The case of the EBSDEs driven by a finite dimensional reflected forward equation together 
with its applications to semilinear PDEs with general Neumann boundary conditions was then 
treated in [20] . 

The main assumption in [12] (and, with slight modifications, in [20J) is the strict dissipativity 
of A + F, i.e., the existence of k > such that for all x and x' in the domain of A, the following 
holds 



Such a requirement ensures the uniform exponential decay of the difference between the trajec- 
tories of two solutions of equation (II. 2|) that plays a crucial role in the arguments in [12J. 

The aim of the present paper is to show that, when G is invertible, we can drop the dissipa- 
tivity assumption on A + F and study (jl.ip when A is dissipative but F is only assumed to be 
bounded and Lipschitz with no restrictions on its Lipschitz constant. See Example 15. 1.11 to com- 
pare, in the concrete case of an ergodic problem for a stochastic heat equation, the assumptions 
needed in the present paper and the ones needed in [12] . 

Our main tool is a coupling estimate for a perturbed version of the forward stochastic differ- 
ential equation (|1 .2|) . Coupling estimates have been recently developed for many different classes 
of stochastic partial differential equations and exploited to deduce regularity properties of the 
corresponding Markov semigroup, see e.g. [7], [15], [16], [22] and [18]. In the present paper, in 
comparison with the previously mentioned literature, we are only dealing with bounded and ev- 
erywhere defined non-linearities but we have to consider measurable non-linearities and to prove 
that the estimate depends on them only through their sup (see Theorem 12.41 and Appendix l6.ip . 

The coupling estimate is used here to get a uniform bound for Yf' a — Yq ,c * where Y x ' a is the 
solution of the following strictly monotonic BSDE (see |21| ) 



Then the Bismut-Elworthy formula for BSDEs (see |10j ) yields the uniform bound for V x Y ' a 
that allows to pass to the limit as a \ in the above equation. Note that the non degeneracy 
of the noise, i.e., the invertibility of G, is used in an essential way at this step. It is also used to 
prove the coupling estimate but a more sophisticated coupling argument, which would not need 
this assumption, could be used. 

We also notice that we construct a "Markovian" solution of the EBSDE in the sense that 
and Zt are deterministic functions of Xf . We prove, by use of the recurrence of the perturbed 
forward stochastic differential equation, that such a "Markovian" solution of the EBSDE is 
unique. The recurrence property is studied in [6] for a forward SDE similar to ours; the difference 
is that here we need to consider drifts that are only bounded and measurable, (see Theorem 2.6 
and Appendix I6.2p . The uniqueness argument is inspired by the corresponding one in |13j . 



(A(x - x') + F(x) - F(x'),x - x') < -k\x - x' 



x,a 




2 



Once existence and uniqueness of a Markovian solution of the EBSDE is proved we can 
proceed as in [12] to deal with an optimal control problem with state equation 

dX?' u = (AX^ U + F(X^ u ) + GR(u t ))dt + GdWt, X*' u = x, (1.3) 

and ergodic cost functional 

1 r T 

J(x,u) = lim sup -E U ' T / L(X x s ,u s )ds. (1.4) 

T->oo T Jq 

Then we deduce that the ergodic Hamilton Jacobi Bellman equation 

Cv(x) + ip (x, Vv(x)G) = A, xeE, (1.5) 

has a unique mild solution; moreover the ergodic problem admits a unique optimal control that 
satisfies an optimal feedback law given in terms of the gradient on the solution to the HJB 
equation (II. 5ft : finally the optimal cost is A. 

In the finite dimensional case there are several papers devoted to the study, by analytic tech- 
niques, of stochastic optimal ergodic control problems and of the corresponding HJB equations 
(see for instance PQ and [2]). On the contrary, to the best of our knowledge, there are very 
few works devoted to the infinite dimensional case. As far as we know (mild) solutions of an 
equation like fjl .5j) was studied, in the infinite dimensional case, only in [13] (besides the already 
discussed results included in [12j). 

In |13| authors prove, by a fixed point argument, existence and uniqueness of the solution of 
the mild stationary HJB equation for discounted infinite horizon costs. Then they pass to the 
limit, as the discount goes to zero. They work under the same non-degeneracy assumption that 
we use here and assume that A is the generator of a contraction semigroup and F is dissipative; 
they also have a limitation on the Lipschitz constant (with respect to the gradient variable) of 
the Hamiltonian function ip (see |11| for similar conditions in the case of a strictly monotonic 
stationary HJB equation ). On the contrary unbounded non-linearities F can be considered in 

The present paper is organized as follows. First we establish general notation. In section [2j 
we introduce the forward equation and state the coupling estimates and recurrence property for 
the perturbed forward equation. The ergodic BSDE is studied in Section [3j Sections H] and [5] 
shortly recall how the previous results can be applied to the ergodic Hamilton-Jacobi-Bellman 
equation and to the ergodic optimal control problem. We include the proofs for the coupling 
estimates and the recurrence property for the perturbed forward equation in the Appendix (see 
section [6} . 

1.1 General Notation 

We introduce some notations; let E, F be real separable Hilbert spaces. The norms and the 
scalar product will be denoted | • |, (■, •), with subscripts if needed. L(E,F) is the space of 
linear bounded operators E — > F, with the operator norm. The domain of a linear (unbounded) 
operator A is denoted D{A). 

Given <j) € Bf,(E), the space of bounded and measurable functions <fi : E — > R, we denote 
\\4>\\o = swp xeE \(f>(x)\. If, in addition, eft is also Lipschitz continuous then ||0||ijp = \\<fi\\o + 

^Px,x'€E,x^x' \<t>{x) ~ 4>{x')\\x - X'\~ l . 

We say that a function F : E — >• F belongs to the class Q 1 {E,F) if it is continuous, has a 
Gateaux differential VF(x) € L(E, F) at any point x E E, and for every k G E the mapping 
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x — > VF(x)k is continuous from E to F (i.e. x — > VF(x) is continuous from E to L(E,F) if 
the latter space is endowed with the strong operator topology). In connection with stochastic 
equations, the space Q 1 has been introduced in [9], to which we refer the reader for further 
properties. 

Given a probability space (f2, J-, P) with a filtration (J-t)t>o we consider the following classes 
of stochastic processes with values in a real separable Banach space K. 

1. Lj,(fl, C([0, T], K)), p € [l,oo), T > 0, is the space of predictable processes Y with 
continuous paths on [0, T] such that 

\ Y \ P L^(n,C([0,T],K)) = E t ^ T] \ Y *\k < °°- 

2. Ly,(tl, L 2 ([0, T}; K)), p £ [l,oo), T > 0, is the space of predictable processes Y on [0,T] 
such that 

( f T V /2 

\ Y \ P L^Lmo,T] ]K )) = E [J o \ Y t\Kdtj < OO. 

3. L 2 v loc (tt;L 2 (0,oo;K)) is the space of predictable processes Y on [0, 00) that belong to the 
space L^(Q, L 2 ([0, T\;K)) for every T > 0. 



2 The forward SDE 
2.1 General assumptions 

This section is devoted to the following mild ltd stochastic differential equation for an unknown 
process X T , r € M + , with values in a Hilbert space if: 

X T = e {T - t)A x+ [ T e (T ~ a)A T(X a ) da + f ^ T ~^ A GdW a , Vr > 0, P-a.s.. (2.1) 
Jt Jt 

We assume the following: 
Hypothesis 2.1 

(i) A is an unbounded operator A : D(A) C H — > if, wii/i -D(^4) dense in H . We assume that 

A is dissipative and generates a stable Co-semigroup {e tA }t>o- By this we mean that there 
exist constants k > and M > such that 

(Ax,x) < -k\x\ 2 VxeD(A); \e rA \ < Me~ kr . 

(ii) For all s > 0, e sA is a Hilbert- Schmidt operator. Moreover \e sA \L 2 ^f{,H) ^ L s 7 f or suitable 

constants L > and 7 G [0, 1/2). 

(iii) T is a bounded measurable map H — > H , 

(iv) G is a bounded linear operator in L(3,H). Moreover we assume that G is invertible and 
we denote by G~ l its bounded inverse. 

(v) (fi, J 7 , P) is a complete probability space, (Ft)t>o is a filtration in it satisfying the usual 

conditions and (Wt)t>o is an T- cylindrical Wiener process with values in a separable Hilbert 
space H. 
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Remark 2.2 We notice that if the operator A with dense domain is m-dissipative that is 
(Ax,x) < — Vx £ D(A) and ^4 — k\I is surjective for a suitable k\ > then by the 
Lumer-Phillips theorem it follows immediately that A generates stable Co-semigroup of contrac- 
tions (that is M = 1). 

The following result is well known in its first part (see, for instance, [5]) and a straight-forward 
consequence of the Girsanov transform in the second. 

Proposition 2.3 Fix t > and x E H and assume that T is Lipschitz. Under the assumptions 
of Hypothesis \2.1\ there exists a unique adapted process X verifying &2.1\) . Moreover, for every 
p € [2,oo) and every T > t, X £ L^(Q; C([0, T]; H)) and 

E sup \X T \ P < C(l + |x|) p , (2.2) 

re[t,T] 

for some constant C depending only on p,^,M and sup x( z H |T(x)| but independent ofT>t. 

If T is only bounded and measurable, then the solution to equation \2. 1\) still exists but in 
weak sense. By this we mean, see again JE/, that there exists a new J 7 -Wiener process (Wt)t>o 
with respect to a new probability ¥ (absolutely continuous with respect to ¥), and an J- -adapted 
process X with continuous trajectories for which \2. 1\) holds with W replaced by W . Moreover 
\2.2\) still holds (with respect to the new probability). Finally such a weak solution is unique in 
law. 

In the following we will denote the solution of equation (|2.ip by X ,x and by X x when we 
choose the initial time t = 0. We remark that equation (12. ip is the mild version of the Cauchy 
problem: 

f dXi'* = AX t s ' x ds + T(X t s > x )dt + GdW s , 8 >t, , OQ , 
I X t = x - 
The following result is proved in section 6.1. 

Theorem 2.4 (Basic coupling estimate) Assume that T : H — > H is Lipschitz and let X x 
be the (strong) solution of equation h2.1\) then there exist c > and fj > such that for all 

<peB b (H) 

\VtMx) - V t [<t>](x')\ < c(l + \x\ 2 + \x'\ 2 )e-^\ 0l (2.4) 

where Vt[<p]{x) = ~K(j){Xf) is the Kolmogorov semigroup associated to equation 12. 
We stress the fact that c and fj depend on T only through sup xg # |T(x)|. 

Corollary 2.5 Relation \2.J$ can be extended to the case in which T is only bounded and 
measurable and there exists a uniformly bounded sequence of Lipschitz functions {T n } n >\ (i.e. 
Vn, T n is Lipschitz and sup n sup a , |T n (x)| < oo) such that 

limT n (x) = T(x), Vx 6 H. 

n 

Clearly in this case in the definition of Vt[4>] the mean value is taken with respect to the new 
probability P. 

Proof. It is enough to show that if V n is the semigroup corresponding to equation (|2.ip with 
T replaced by T n , then Vx € H and Vi > 0, 

vn<i>](x)^v t [<t>}(x). 
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We set 

U* = e rA x + I e^ A G dW a . 
Jo 

By Girsanov's formula 

vm( X ) = e( P ^(e?)), vtMx) = nPtHun) 

where 

n,x 
Pt 



and 



exp (- j\G- x T n {U?),W s ) B -\J^ \G- l r n (U?)gds) , 
% = exp (- J'\g~ 1 T(U x ), dW s )s - \ j* \G- l T{Uf)\lds^ . 



We have 



E 



exp (-2 jlG-^Uf^dW^E - 2^ < |G- 1 T n ([/nilds) exp (^J^G^T^U^lds 



< exp ( t\G 1 \ 2 sup sup |T n (x)| 2 ) < oo, 

n x 



from which we deduce that {Pt' x } n is uniformly integrable in L 1 (il). Moreover, it is easy to see 
that lim n p™' x = pf in probability, and the claim follows. □ 



The equation (|2.ip also enjoys a recurrence property that will be useful in the following, it 
is proved in section 6.2. 

Theorem 2.6 Assume that T : H — > H can be approximated (in the sense of pointwise conver- 
gence) by a uniformly bounded sequence of Lipschitz functions {T n } n >i. Then the solution of 
equation $2.1\) is recurrent in the sense that for all V £ H , V open: 

lim F{3t € [0,21 : Xf G T} = 1. 

T->oo 

In particular, setting t x = inf{i : \Xf\ < e}, then Ve > 0, lirnr^oo Pjr 1 <T} = 1. 



3 The Ergodic BSDE 

We fix now a bounded function F : H — > H and denote by X t,x (and by X x when we choose 

the initial time t = 0) the solution of equation (]2.ip with T = F. 

This section is devoted to the following type of BSDEs with infinite horizon 

Y? = Yt + [ T [HXa, - A] da - / T ^ dW CT , < t < T < oo, (3.1) 
Jt Jt 

where A is a real number and is part of the unknowns of the problem; the equation is required 
to hold for every t and T as indicated. On the function : H x H* — > R and F we assume the 
following: 

Hypothesis 3.1 ip is a measurable map H x S* — > R. Moreover there exists I > suc/i f/iai 
|^(aj, 0)| < /; [*0(a;, z) — ^(x, z')| < i|z — z r \, x £ H, z, z' E E* . 
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Hypothesis 3.2 F is bounded, Lipschitz and Gateaux differentiable, more precisely, F belongs 
to the class Q 1 ^,!!). 

We start by considering an infinite horizon equation with strictly monotonic drift, namely, for 
a > 0, the equation 

y x,a = y x,a + [ T ^( X *, Z x > a ) - aY*> a )da - [ T Z x > a dW a , < t < T < oo. (3.2) 
Jt Jt 

The existence and uniqueness of solution to (13. 2p under Hypothesis 13.11 was first studied by 
Briand and Hu in [3] and then generalized by Royer in [21]. The following lemma follows from 
Lemma 2.1 and Theorem 3.2 in 1141. 



Lemma 3.3 Let us suppose that the Hypotheses \2A\ \3.1\ and 3.2 hold. Then for all x € H and 

a > 0, 

(i) there exists a unique solution (Y x,a , Z x,a ) to BSDE \3. I| ) such that Y x,a is a bounded 
continuous process, Z x ' a belongs to L 2 ? loc (S^; L 2 (0, oo; H*)), and \Y^' a \ < I /a, F-a.s. for all 
t > 0; 

(ii) if we define v a {x) = Yq' then, for all fixed a > 0, v a is Lipschitz bounded and of class 
Q l , moreover, 

Y t x ' a = v a (X?), Z?' a = Vv a (X?)G. 

In order to construct the solution to (13. ip . we need some uniform in a estimate of \v a {x) — 
v a (x')\. This will be obtained by coupling estimates but first we have to prove an approximation 
lemma: 

Lemma 3.4 Let £, (' : H — > H* weakly* continuous with polynomial growth. We define 

ip(x,((x)) - i/j(x,('(x)) 



T(s) 



\C(x))-C'(x)\ 2 
0, if ((x) = C(x). 



(C(x)-C'(x)r, if ((x)^('(x), 



There exists a uniformly bounded sequence of Lipschitz functions (T n ) n >i (i.e., Vn, T n is Lips- 
chitz and supyjSup^ |T n (x)| < oo) such that 

lim Y n (x) = T(x), Vx G H. 

n 

Proof. Fixing an orthonormal basis {£i,£2 ; ■ • ■ } in S, we define the projection n Pi s*: H* — > H* 
as follows: 

n 

n p ,E*C = ^(C&) <&,->• 

Set 

It is easy to verify that the functions T*' p are continuous functions. Moreover |T*' p (x)| < I and 
lim p T i,p (x) = T*(x) , limj T*(x) = T(x), for all x E H. Fixing i,p, it is quite classical (based on 
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finite dimensional projections and convolutions) to construct a uniformly bounded sequence of 
Lipschitz functions {Y*' p ' m } m , such that lim m T*' p,m (x) = T l,p (x), see, e.g. Lemma 4.2 in [TO] . 
Then the proof ends with a diagonal procedure. □ 



The following lemma plays a crucial role. It gives the desired estimate of v a {x) — v a (x') and 
of Vv a . 



Lemma 3.5 There exists a constant c(£, c,rj) > such that for all x, x' € H 

\v a (x) -v a (x')\ <c(l + |x| 2 + |x'| 2 ); 

and for all x 6 H, 

\Vv a (x)\ < c(l + |x| 2 ). 

We stress the fact that c > is independent of a. 
Proof. Set 

' ip(x,Vv a (x)G) -ip(x,0) 



(3.3) 
(3.4) 



f a (x) = < 



(Vv a (x)G)* , if Vv a (x)G ^ 



Then 



\Vv a {x)G\ 2 
0, if Vv a (x)G = 0. 



MX?, zr) = ^(Xf,0) + ? a (x?)z?' a . 



From Proposition 13.41 T a is the pointwise limit of a uniformly bounded sequence of Lipschitz 
functions. 

For all T > 0, the couple of processes (Y x ' a , Z x,a ) is a solution to the following finite horizon 
linear BSDE 



-dY*' a = i){Xf,Q)dt + T a {Xf)Z x t ' a dt - aY^dt - Z x t ' a dW u t € [0, T], 
Y*' a = v a (X x ). 

Since T a is bounded for all T > 0, there exists a unique probability P x ' a > T such that 

,-t 



(3.5) 



T a (X x )ds + W t 



o 



is a ]P x > a > T - Wiener process for t £ [0, T]. Consequently we have 



v a (x) = E 



x,a,T 



e~ aT v a {X^)+ I e- as ij;(X*,0)ds 



where K x,a,T denotes the expectation with respect to p x > a > T . 
Letting T — >• oo, as |v a (x)| < K we get 



v a (x) = lim t x ' a > T 

T->oo 



e- a ^(I s J ,0)ds 



On the other hand, if we rewrite the forward equation (|2.ip with respect to W x ' a it turns out 
that X x verifies 



dXf = AXfdt + F(Xf)dt + GT a {Xf)dt + GWf 

L 



(3.6) 
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We denote by V a the associated Kolmogorov semigroup, i.e., 

vm(x) = ^ t 4>{xf). 

Applying Corollary 12.51 with T a = F + GT a (which is also the pointwise limit of a sequence of 
Lipschitz functions), we obtain 

poo ~r 

\v a {x)-v a {x')\ < / e- at \V?[if;(-,0)](x) -V?[i>(;0)](x')\dt < "(1 + \x 2 \ + |x'| 2 ) 
Jo ' 1 1] 

where c and fj are independent of a. The proof of (|3.3|) is now complete. 
To prove (|3.4|) . let us set 

v a (x) = v a (x)-v a (0). 
Then, Y x ' a = Y^' a — = v a {Xf ) is the unique solution of the finite horizon BSDE 

-dY t x ' a = iP(X? , Z x ' a )dt - aY t x ' a - av a (0)dt - Z x ' a dW u 
Y x ' a = v a {Xf). 

Note that in particular, in the above equation, |cw a (0)| < /. By Theorem 4.2 in [lOj . v a is of class 
Q l and there exists a constant c(l,c,fj) > independent of a such that iVt/^x)! < c(l + \x\ 2 ), 
and the conclusion follows. □ 



Remark 3.6 As already mentioned in the introduction, the non degeneracy assumption on G 
is essential in the proof of the gradient estimate on v a . More precisely, it is necessary to use the 
Bismut-Elworthy formula from 

Now we are in position to state our main result in this section. 

Theorem 3.7 Assume that the Hypotheses \ 2.1l \3.1\ and 3.2 hold. Moreover let A be the real 
number in \3. 7\ ) below and define Y t x = v(Xf) (where v is a locally Lipschitz function with 
v(0) = defined in i3.8\) ). Then there exists a process Z £ L^, loc (r2; L 2 (0, oo; H*)) such that 
F-a.s. the EBSDE (KM is satisfied by (Y x , Z x , A) for allO<t< T. 

Moreover v is of class Q , \Vv(x)\ < c(l + \x\ 2 ), and Z t = Vv{Xf)G. 



Proof. Let us set again v a {x) = v a {x) — v a (0). By Lemma I3.3I and relation (|3.3|) we can 
construct, by a diagonal procedure, a sequence a n \ such that for all x in a countable dense 
subset D C H 

v an (x)^v(x), a n v an {0)^\, (3.7) 

for a suitable function v : D — > K and for a suitable real number A. 

Moreover, by Lemma I3.5I \v a (x) — v a (x')\ < c(l + \x\ 2 + |x'| 2 )|x — x'\ for all x,x' G H and 
all a > 0. So v can be extended to a locally Lipschitz function defined on the whole H with 
\v(x) — v(x')\ < c(l + \x\ 2 + |x'| 2 )|x — x'\ and 

v an {x) ->■ v(x), xeH. (3.8) 

Clearly we have, P-a.s., 
_ _ i-t _ 

y x,a = y x,a + / ^ X x^ Z x,aj _ a y x > a _ av <* (Q))da - / Z X ^ a dW a , < t < T < 00. (3.9) 
Jt Jt 
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Since |^ Q (a;)| < c(l + \x\ 2 ), inequality (|2.2p ensures that Esup te r 0)T i [sup a>0 |y^' a | 2 ] < +00 for 
any T > 0. Thus, if we define Y x = v(X x ), then by dominated convergence theorem 

r T — — 

I \Y x t ' an -Y x t \ 2 dt^0 and E|y^ n - Y X T \ 2 -> 



as n — > 00 (where a n \ is a sequence for which (|3.7p and fj3.8j) hold) . 
We claim now that there exists Z € loc (fi; £ 2 (0, 00; 3*)) such that 

E / \z^ an -z x \Ldt ->0. 

•/ 

Let y = y x ' a " — y x ' Q ™ ) £ = Z a ' ,an — Z x,am . Applying Ito's rule to Y 2 we get, by standard 
computations, that 

y 2 + E f \Z t \Ldt = EYS + 2E f fcYtdt - 2E f [a n Y x ' a " - a m Y x ' am ] Y t dt, 
Jo Jo Jo 

where fa = if>(X? , Z^ an ) - ^(Xf, Z*' am ). We notice that \fa\ < i|Z t | and a n \Y x ' a "\ < I. Thus 

rT r /-T n 

; / |Zt||„dt<c E(y T ) 2 +E / (Y t ) 2 dt + E / |y t |dt . 

Jo Jo 



E 



It follows that the sequence {Z x,a ™} is Cauchy in L 2 (Q; L 2 (0, T; 3*)) for all T > and our claim 
is proved. 

Now we can pass to the limit as n — > 00 in equation (13. 9p to obtain 



Y x t =Y x T + £ (4>{X*,Z x a )- \)da- £ Z x a dW a , 0<t<T<oo. (3.10) 

We notice that the above equation also ensures continuity of the trajectories of Y. 

Finally, the couple of processes (F ,Z ) is the unique solution of the finite horizon BSDE 

-dYf = &(X?,T t ) - *)dt - ZfdW t , 

yf = v(xf). 

Once again, by Theorem 4.2 in [10], we conclude the proof. □ 

Remark 3.8 The solution we have constructed above has the following "quadratic growth" 
property with respect to X: there exists c > such that, P-a.s., 

\Y X \ < c(l + \Xf\ 2 ), for all t > 0. (3.11) 

If we require similar conditions then we immediately obtain uniqueness of A. 

Theorem 3.9 Assume that the Hypotheses \2.1\ \3.1\ and 3.2 hold true. Moreover suppose that, 
for some x € H, the triple (Y',Z',\') verifies P-a.s. equation \3.1\) for all < t <T, where 
Y' is a progressively measurable continuous process, Z' is a process in Lp i oc (f2; L 2 (0, 00; 3*)) 
and A' 6 M. Finally assume that there exists c x > (that may depend on x) such that for some 
p>l, P-a.s. 

\Y(\ < c x (l + \Xf\ p ), for all t > 0. 

Then A' = A. 
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The proof of the above theorem is similar to that of Theorem 4.6 in |12| . so we omit it here. 

The solution obtained in Theorem 13. 71 has moreover the property that processes Y x and Z x 
are deterministic functions of X x . We refer to such solutions as to "Markovian" solution of the 
EBSDEs. 

We prove that the Markovian solution is unique. 

Theorem 3.10 Let (v,C); (^>C) two couples of functions with v, v : H — >■ R, continuous, with 
\v(x)\ < c(l + \x\ 2 ), \v(x)\ < c(l + |x| 2 ), v(0) = v(Q) = and C continuous from H to H* 
endowed with the weak* topology verifying \C(x)\ < c(l + \x\ 2 ), \(,{%)\ < c(l + |x| 2 ). 

Assume that for some constants \, A and all x £ H, (v (Xf ), Q(X X ), A), (£(Xf ), £(Xf ), A) 
ven/y toe EBSDE (E2P, i/iera A = A, w = u, C = C 



Proof. The equality A = A comes from Theorem! 

Then let Y t x = v{Xf) - v(Xf), Z'f = Q{Xf) - Q(Xf) and f be defined in Proposition EQ 
We have 

-dYf = f(Xf)Zfdt - ZfdWt = -Z x dW[ 

where W[ = - f* T(X x )ds + W t is a Wiener process in [0,T] under the probability W x > . 

Moreover, under ¥ X,T , X x satisfies equation (|2.3p . in [0, T], with, as before T = GT + F. 
Thus, from Proposition 12.31 it holds that for all p > 1, and all x 6 H 

E x ' T \Xf \ p < c(l + \x\ p ), V0 <t<T, 

where c > depends on p, 7, M and Z|G| + sup^ |F(a;)|, and is independent of T. Thus the 
growth conditions on £ and £ implies that, for all T > 0, E^' 7 " J"^ |Zf | 2 di < 00. 
Let r = inf{t : \Xf\ < e} then for all T > 

r — ie, i T hT- 

For any 5 > 0, there exists e > such that \v(x) — v(x)\ < S if [ar| < e. Then for a constant 
O0, 

= \E x > T Yf AT \ < E X > T \Y?\1 {T<T} +E x ' T \Yf\l {T > T} 

< 5+{F x ' T {t > T}) 1 ' 2 (E x ' T {\Yf\ 2 }) 1/2 

< 5 + (F x ' T {t > T}) 1/2 (E X ' T {1 + \X X T | 4 }) 1/2 . 

Noting that, by Theorem l2.6l limj'_ i . 00 F x,t {t > T} = and sending T to 00 in the last inequality, 
we obtain that \Yq | < & and the claim follows from the arbitrarity of 5. □ 



4 Ergodic HJB equations 

We briefly show here that as v(x) = Yq in Theorem 13.71 is of class Q l , the couple (v, A) is a mild 
solution of the following "ergodic" Hamilton-Jacobi-Bellman equation: 

Cv(x) + ip (x, Vv(x)G) = A, xeH, (4.1) 

where the linear operator C is formally defined by 

Cf (x) = l -Trace (GG*V 2 f (x)) + (Ax, V/ (x)) + (F (x) , V/ (x)). 
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We notice that we can define the transition semigroup (Pt)t>o corresponding to X by the formula 
Pi[^](x) = E(ft{Xf) for all measurable functions (ft : E —> M having polynomial growth, and we 
notice that C is the formal generator of (Pt)t>a- 

Since we are dealing with an elliptic equation it is natural to consider (v, A) as a mild solution 
of equation (|4.1|) if and only if, for arbitrary T > 0, v[x) coincides with the mild solution u(t, x) 
of the corresponding parabolic equation having v as a terminal condition: 

+ Cu (t, x) + V (x, Vu (t, x) G) - A = 0, t E [0, T], x E H, 
< ' (4.2) 
u(T, x) = v(x), x € H. 

Thus we are led to the following definition: 

Definition 4.1 A pair (v, A) (v : H — >• R and A € M,) is a miW solution of the Hamilton- J acobi- 
Bellman equation f^. J| ) i/ f/ie following are satisfied: 

1. ve gHH,^); 

2. there exists C > suc/i f/iaf |Vv (x)| < C (1 + |x| p ) /or every x E i? and some p > 1; 

3. for < t < T and x £ H, 

v(x) = Pr_ t [v] (x) + ^ (P s _ t [V>(-, Vv (•) G)] (x) - A) ds. (4.3) 

Theorems 13.71 and 13.101 immediately yield existence and uniqueness of the mild solution of 
equation (14. ip . 

Theorem 4.1 Assume that Hypotheses \2. 1\ and \3. 1\ hold. 

Then (y, A) is a mild solution of the Hamilton-, J acobi- Bellman equation j^. 

Conversely, if(v, A) is a mild solution of |^.1[ ) i/ien, setting Y x = v{Xf) and Zf = Vv{Xf)G, 
the triple (Y X ,Z X ,X) is a solution of the EBSDE \3.1\) , which implies the uniqueness of mild 
solution in the sense that if (v l , A), i = 1,2 are mild solutions of the Hamilton- J acobi- Bellman 
equation g^j then v 1 {X x ) = v 2 (X x ) and Vv 1 {X x )G = Vv 2 {X x )G P- a.s. for a.e. t > 0. 

Proof. : The proof is identical to the one of Theorem 6.2 in [12]. 



5 Ergodic control 

We fix a bounded function F : H — >• H and denote by X x the solution of equation (12. ID with 
T = F. 

Assume that the Hypotheses 12. II and 3.2 hold. Let U be a separable metric space. We define 
a control u as an (^^-progressively measurable [/-valued process. The cost corresponding to a 
given control is defined in the following way. We assume that the functions R : U —¥ H* and 
L : H x U — > M are measurable and satisfy, for some constant c > 0, 

\R(u)\ < c, |L(x,u)|<c, |£(x, u) — L(x , u)\ < c |x — x |, uGU,x,x'gH. (5.1) 

Given an arbitrary control u and T > 0, we introduce the Girsanov density 

# = exp R(u s )dW s \R(u s )\Lds^ 
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and the probability P^ = p^.P on Tt- The ergodic cost corresponding to u and the starting 
point x € H is 

1 f 7 " 

J(x,u) = lim sup -E U ' T / L(X x s ,u s )ds, (5.2) 

where E n,T denotes expectation with respect to Py. We notice that W" = W t — J* * R{u s )ds is a 
Wiener process on [0, T] under Pj, and that 

dXf = (AATf + F(Xf))dt + G(dW t u + R(u t )dt), t e [0, T] 

and this justifies our formulation of the control problem. Our purpose is to minimize the cost 
over all controls. 

To this purpose we first define the Hamiltonian in the usual way 

ip(x,z) = inf {L(x,u) + zR(u)}, x£H,z£E*, (5.3) 

and we remark that if, for all x, z, the infimum is attained in (|5.3p then by the Filippov Theorem, 
see [17], there exists a measurable function 7 : H x H* — >■ U such that 

ip(x, z) = l(x, j(x, z)) + zR(j(x, z)). 

We notice that under the present assumptions ip is a Lipschitz function and ^(-,0) is bounded 
(here the fact that R depends only on u is used). So if we assume the Hypotheses 12.11 and 3.2, 
then in Theorem 13.71 we have constructed, for every x G H, a triple 

(Y*,Z*,\) = (v(X x ),aX*),\) (5.4) 

solution to the EBSDE (1570 . 

Theorem 5.1 Assume that the Hypotheses \2.1\ and 3.2 hold, and that \5.1\) holds as well. 

Moreover suppose that, for some x £ H, a triple (Y,Z,X) verifies F-a.s. equation h3. 1\) for 
all < t < T, where Y is a progressively measurable continuous process, Z is a process in 
L^, loc (fi;L 2 (0,oo;H*)) and AeR. Finally assume that there exists c x > (that may depend on 
x) such that P-o.s. 

\Y t \ < c x (l + |Xf | 2 ), /or a// i > 0. 

Then the following holds: 

(i) For arbitrary control u we have J(x,u) > A = A, and the equality holds if L(Xf ,Ut) + 
ZtR(ut) = ip{Xf ,Zt), P-a.s. for almost every t. 

(ii) If the infimum is attained in \5. 3\) then the control u~t = y(X£,Zf) verifies J(x,u) = A. 

In particular, for the solution |5.^[ ) mentioned above, we have: 

(Hi) For arbitrary control u we have J(x,u) = A if L{Xf ,ut) + (,{Xf )R{u t ) = ip(Xf ,((X£)), 
P-a.s. for almost every t. 

(iv) If the infimum is attained in \5. 3\) then the control ut = j(Xf,^(Xf)) verifies J(x,u) = A. 
Proof. : The proof is identical to the one of Theorem 7.1 in [12] . 
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Example 5.1.1 We consider here an ergodic optimal control when the state equation is a 
stochastic heat equation. The difference with respect to the same example in J7||/ is that, if 
we have non- degenerate noise, we do not need to assume that the non-linearity f is decreasing. 
Namely we consider the following state equation 

' d t X u (t, = [j^X u (t, + f (£, X u (t, 0) + r(£, u (t, 0)} dt + a(OW (t, £) dt, 
X u (t,0)=X u (t,l)=0, ( 5 - 5 ) 

[ x u (t,o = x (O, 

where W(t,£) is a space-time white noise on [0, oo[x [0,1]. 
We also introduce the cost functional 

J(x,u) = limsup^E f f 'l(Z,XX(Z),u a (Z))dtds. (5.6) 

T->oo 1 JO JO 

An admissible control u(t,£) is a predictable process u : f2 x [0,oo[x[0, 1]. 

Then ( the reduction to the abstract infinite dimensional framework is as in U(J\j Section 5.1) 
the results of Theorem I5.il can be applied under the following assumptions: 

1. f : [0, 1] x R — y R is measurable and bounded and 

\f(£,m) - f(£,,m)\ < Cf\v2 - ml 

for a suitable constant cf, almost all £ £ [0,1], and all r\\,r\2 € R. Moreover we assume 
that /(£, •) G C^R) for a.a. £ G [0, 1]. 

2. a : [0, 1] — > R is measurable and bounded. Moreover c a < |<r(^)|, for a.a. £ € [0,1] and a 
suitable constant c a > 0. 

3. r : [0, 1] x R is measurable and bounded and, for a.a. £ € [0, 1], the map r(£, •) : R — > R is 
continuous. 

4- I [0, 1] x R 2 is measurable and bounded and, for a.a. £ G [0, 1], the map /(£, •, •) : R 2 — > R 
is continuous. 

5. x £ L 2 ([0,1]). 



6 Appendix 

6.1 Proof of the coupling estimates 



Proof. We use a coupling argument. The precise result needed here cannot be found in the 
literature and we give the full proof for completeness. In particular, we show that the constant 
c and fj in (|2.4h depend only on the supremum of T. Our proof is a mix of arguments found in 
[7], [15], [16], [22] and [18]. Note that the more analytical method from [1] could also be used. 

To prove that the laws of the solutions starting from different initial data get closer in 
variation, we first wait that these solutions enter a fixed ball. Then, we construct a coupling of 
solutions starting from initial data in this ball. Iterating this argument we obtain the result. 

In this section, Ki, i = 0, 1 . . . denotes a constant which depends only on j,M,G and 
L = sup xeH \F(x)\. 
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Step 1: Let Z t = J * e^'^GdW^) and p = X x - Z. Thanks to Hypothesis EH we obtain 
by taking the scalar product of the equation verified by p with p: 

I4|p| 2 + A;H 2 <LH<^ + -| /9 | 2 
2dt lHl IPI ~ IPI ~ 2k 2 in 

with Lq = sup x£H |T(x)|. By Gronwall's lemma 



\pt\ 2 <e- kt X\A 2 + §) 



Moreover 



H\Zt\ 2 ) = jT \e Ait - s) G\l 2{H;H) d S < \G\l (H) jf* |e As || 2(/ , ;H)( i S . 



It follows 

E|Xf| 2 < 2(\x\ 2 e- kt + «i), (6.1) 

with a constant «i depending only on 'y, iW, G and Lq — sup^gj^ l-F^x)] but independent of t > 0. 
By the Markov property: 

E (I^+i)t! Vfcr) < 2(|^ T | 2 e- fcT ) + «i, fc > 0. (6.2) 

Let us define for R > 

Cfc = {|Xfc T | 2 > R}, B k = Dj =0 Cj. 

By Chebychev's inequality 

nC k+ i\F kT ) < ~^-\Xh\ 2 + f ■ (6-3) 
Multiply (|6.2p and ()6.3[) by l_B fe and take the expectation to obtain, since l_B fe+1 < ls fc , 

i(ix- +1)T i 2 i Bfe+1 ) \ / n\xi T \ 2 i Bk 

P(s fc+ i) / V P(B fc ) 

where 

/ 2e" fcT ki \ 



2e- fcT asi 
V R ~R J 



Let i? = Ak\ and choose T such that e = g. The eigenvalues of A are and 2e + 4 < g. 
We deduce that 



k 



ns fc )<« 2 (gj a + M 2 ), 

with K2 depending only on «]_. Defining 

r = inf{A:T;|^ r | 2 <i?}, 

it follows 

P(t > kT) < F(B k ) < k 2 (-) (1 + |x| 2 ). 
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Thus, for r]T < 2 In 2, 

E(e^ T ) < /c 3 (l + M 2 ). (6.4) 

Step 2: We construct a coupling for x, y £ -Br, the ball of center and radius R, x ^ y. 
Let T > to be chosen below, we denote by /Ui the law of X x and //2 the law of X y on [0,T]. 
Set X t = Xf + ?f±e At (x - y) and denote by fi 2 the law of X on [0, T]. Then 

dX = (AX + F(X))dt + dtif, 

where W t = W t — J * h(s)ds, with /i(s) = F(X S ) — F{Xi) — ^e At (x — y). By Girsanov's formula, 
If is a Wiener process under a new probability measure P. Therefore, under P, X has the law 
Hi while under P it has the law jl 2 . Of course Hi an d A*2 are equivalent. Since \h(t)\ < 2L + 2^, 
we deduce that 

1 ( I d Hi - K 4- 



We need the following result (see for instance 
Proposition 6.1 Let (^1,^2) be two probability measures on a Banach space E then 

II A* 1 _ M2IITV = minP(Zi / Z 2 ) 

where the minimum is taken on all coupling (Z1 , Z 2 ) o/(/xi, ^2)- Moreover, there exists a coupling 
which realizes the infimum. We say that it is a maximal coupling. It satisfied 

F(Zi = z 2 , Zi e r) = m a M2 (r), r e £(£). 

Moreover, if Hi & n d Hi are equivalent and if 

p) P+1 d,i<c 

dux J 

for some p > 1 and C > 1 i/ien 

l / l \ 

¥(Zi = Z 2 ) = Hi A ^(S) > (1 - -) ( 

We deduce the existence of a coupling (y 1,x,y ,V 2,x,y ) of (hi,H 2 ) such that 

1 



4k 4 ' 

Clearly, (V^ 1 '^'^, V^ 2 ' 2 ^ = t^ 2 '^ _ Z^p*^*^ — y)) 4g j T ] is a coupling of (/Ui,/^ 2 ) on [0, and 

P^'*'* = Vt' x ' v ) > ¥{V x ^ y = V 2 ' x ' v ) > (6.5) 

Step 3: We now construct a coupling for any initial data. For x = y, we set 

(V t 1 ' x > x ,V t 2 > x > x ) = (X?,X?),te[0,T]. 



Recall that if /ii,^2 are absolutely continuous with respect to a measure (for instance fi = fil + /Ua), we 
have 

d(MiAM2) = (^A^)d/,. 
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If x or y is not in Br, we set 



(v t i ™,v?> x > y ) = (x?a?), t€[o,n 

where X y is the solution of equation (|2.3p driven by a Wiener process W independent of W. 
The coupling of the laws of X x , X y for t > is defined recursively by the formula 



v l,X,y _ T 7" ' tiT > V nT 

V nT+t — V t 



,te[0,T], 1 = 1,2. 
We then define the following sequence of stopping times: 



L m = inf{/ > L m _i, V^' v , V% x ' y G 
with Lo = 0. Evidently, (|6.4p can be generalized to two solutions and we have: 

E(e" LlT ) < k 3 (1 + \x\ 2 + \y\ 2 ) 



and 

It follows 
and 

Set now 



ri(L m +i—L m )T 



T Lm r) < /*(! + |V£W + |V*™'* 



E(e 



< K 3 (l + 2i? 2 )E (e^™ 7 ) 
E (e^"^) < 4(1 + 2R 2 ) l - 1 {\ + \x\ 2 + \y\ 2 ). 



inf-f/ V^' x ' y - V z ' x ' y \ 



Since V L 'jt v = V 2 'jt y G .Br, we have by ()6.5[) 

p(4>i + iko>0<(!-r-) 

Since F(£ > I + 1) = P(4 > i + l|4 > F (4 > 0> we obtain 

p(4 > z) < (i - _Ly. 

Then for 7 > 

E(e^o r ) <^E(e^ T l z= , ) 



l>0 
2>0 



4) 1 -T/' 7 (E(e ,?L ' T )) 7/r ' 



< 



El 1 



2>0 



We choose 7 < r] such that 



4k 4 / 



4k 4 / 



(J-l)(l- 7 /„) 



4(l + 2i? 2 y- 1 (l + |xr + |y| i 



7/1? 



1-7/17 



[ K -3(l + 2i? 2 )] 7/,? <l 
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and deduce 
Since 
it follows 
and 

Moreover 



E(e 7 ^o T ) < k 5 (1 + \x\ 2 + \y\ 2 ). 
n = m£{k, V^ x ' y = V 2 ^ y } < L £o + 1 
E(e^ n ° T ) <k 5 (1 + \x\ 2 + \y\ 2 ) 
F(V^ x > y + V 2 ™) = ¥{k > n ) < « 5 (1 + \x\ 2 + M 2 )e" 7fcT 



nv%& + < V(V#* + V%*) < «b(1 + \x\ 2 + |y| 2 )e-^ 

<K 6 (l + [x| 2 + | 1/ | 2 ) e -T(« T +*). 

We deduce for <j) E B b {T d ) with sup^jj |</>(x)| < 1 

WMix) - VtMv)\ = |i(<K^ ,2/ ) - m 2 ' x ' y )) 

< 2k 6 (1 + \x\ 2 + |y| 2 )e~ 7 *. 
6.2 Proof of the recurrence 

Our method consists in applying Proposition 3.4.5 in [6J, and we apply Doob's Theorem (see 
Theorem 4.2.1 in [BJ) in order to verify the conditions of Proposition 3.4.5 in [6]. 

Proof. 

Let us first introduce an auxiliary Markovian semigroup 1Zt[4>](x) = Kcf)(U^) corresponding 
to the Markov process U where 



□ 



■t 

We denote 



U* = e tA x+ [e {t - s)A GdW s 
Jo 



K(t, x,T) = nUf E r) = Tit [l r ] (x), T E B(H) , 

the transition probabilities corresponding to U. 

By Hypothesis 12. 1\ the Markovian semigroup 1Z admits a unique invariant measure v (see 
Theorem 6.3.3 in [B]). Moreover, 1Z is irreducible (i.e., lZ(t,x,T) > for all T which is open and 
non empty set in H, see Theorem 7.2.1 in [6]), and i-regular for any t > (i.e., the measures 
{TZ(t,x, •) : x E H} are equivalent, see Theorem 7.3.1 in |6j). 

Next, recall that Vt[(f>]{x) = K(f>(X£) (where E means expectation with respect to probability 
P) is the Markovian semigroup corresponding to weak solutions X x of the equation (j2.3j) (we 
notice that the solutions of equation (|2.3p are unique in law). And we denote 

V(t,x,F) =P(Xf ET) =V t [l r ](x), FEB(H) 

the transition probabilities corresponding to X. From the Girsanov theorem, the semigroup Vt 
can be represented, for t < T, by 

PtMx)=E(ft<l>(V?)) 
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where p x T = exp (- f Q {G~ l T {Uf) , dW r s ) s - \ jj {G^T^U? > 0, P-a.s. 

Consequently, P is irreducible (i.e., "P^a^r) > for all V which is open and non empty set 
in H), and t-regular for any t > (i.e., the measures {V(t,x, •) : x € -ff} are equivalent). And 
the above representation also implies that the semigroup V is stochastically continuous (i.e., 
hm t \aVt[4>](x) = 4>{x) for all x G H and 4> € C b {H)). 

Moreover, by a similar argument to that of Theorem 8.4.4 in [6], there exists a measurable 
function r/ : H — > IR + such that /i, = ndv is the unique invariant measure corresponding to 
semigroup (Vt)t>o- Indeed it is clear from the proof of Theorem 8.4.3 in [6] that Theorem 8.4.4 
in [6] remains true whenever F can be approximated by a sequence of functions converging 
in the bounded pointwise convergence sense. 

We are now in position to apply Doob's Theorem (see Theorem 4.2.1 in [6]) to obtain that 

• is strongly mixing and Vt(x,T) — > fi(T) for all T € B(H). 

• is equivalent to all measures Vt{x,-). 

Let T be an open and non empty set in H. As \x is equivalent to all measures Vt{x, •) and 
V is irreducible, fj,(T) > 0. Therefore, liminf^oo Vt(x, T) — > fi(T) > 0. By Proposition 3.4.5 in 
[6], the Markovian semigroup V is recurrent and ¥{3t > : Xt G T} = 1. 

In particular if, for all T > 0, X x is a weak solution of equation (12 .3D in [0,T], under the 
probability Fj,, and r = inf{i : \Xf \ < e} then 

lim P£{r x <T}= lim FU inf LX?| < e} = lim P{ inf |Xf I < e} = 1. □ 

T->oo T^oo 0<i<T T^oo 0<t<T 
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